The International Planning Competition Series and Empirical Evaluation of AI Planning Systems
نویسندگان
چکیده
In this paper we consider the role of the International Planning Competition series in the evaluation of planners, both directly through the events themselves, and indirectly through the creation of resources and infrastructure. We also consider the problem of evaluation based on data collected both in the competitions and otherwise and examine some of the issues that arise in attempting to formulate and test hypotheses around the data.
منابع مشابه
Automating the evaluation of planning systems
Research in automated planning is getting more and more focused on empirical evaluation. Likewise the need for methodologies and benchmarks to build solid evaluations of planners is increasing. In 1998 the planning community made a move to address this need and initiated the International Planning Competition – or IPC for short. This competition has typically been conducted every two years in t...
متن کاملThe AIPS-98 Planning Competition
was invited to take part in the first planning competition, hosted by the Artificial Intelligence Planning Systems Conference, to provide a new impetus for empirical evaluation and direct comparison of automatic domain-independent planning systems. This article describes the systems that competed in the event, examines the results, and considers some of the implications for the future of the fi...
متن کاملThe Fifth International Competition on Knowledge Engineering for Planning and Scheduling: Summary and Trends
We review the 2016 International Competition on Knowledge Engineering for Planning and Scheduling (ICKEPS), the fifth in a series of competitions started in 2005. ICKEPS series focuses on promoting the importance of knowledge engineering methods and tools for automated Planning and Scheduling systems.
متن کاملThe 2014 International Planning Competition: Progress and Trends
We review the 2014 International Planning Competition (IPC-2014), the eighth in a series of competitions starting in 1998. IPC-2014 was held in three separate parts to assess state-of-the-art in three prominent areas of planning research: the deterministic (classical) part (IPCD), the learning part (IPCL), and the probabilistic part (IPPC). Each part evaluated planning systems in ways that push...
متن کاملThe Fourteenth International Conference on Automated Planning and Scheduling (ICAPS-04)
ference series was formed in 2003 by merging two biannual conferences: the International Conference on Artificial Intelligence Planning and Scheduling (AIPS) and the European Conference on Planning (ECP). It covers the latest theoretical and empirical advances in planning and scheduling, from new theoretical frameworks and algorithms to practical implementations in a variety of domains. ICAPS-0...
متن کامل